The present invention relates generally to the field of object storage architecture, and more particularly to using storlet in an erasure code object storage architecture for image processing workload.
Traditional object storage architecture comprises two node groups: proxy nodes, which are used for distributing the load and request handling; and storage nodes which are responsible for writing to the disks/storage subsystems. Traditional object storage architecture serves as a storage unit, and in order for analysis of the data residing in these storage units (i.e., extracting meaningful information from raw data) an additional client or computing node is required.
Storlet architecture (i.e., embedded compute infrastructure built-in object storage) comprises a software engine present within the nodes. The end user must frame the computation algorithm and must deploy or pass it to this engine as a normal object PUT operation. Storlet architecture does not require any additional client or compute node to perform analysis of the data, but rather the proxy/storage node itself acts as a compute node and returns the results back to the user. Storlet architecture uses virtual machines (VM) (e.g., Linux Containers, Dockers, KVM, ZeroVM, etc.) deployed on the nodes to perform the computation tasks.
Erasure coding is a method of data protection in which data is broken into fragments, expanded, and encoded with redundant data pieces and stored across a set of different locations or storage media.